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ABSTRACT 
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collecting evidence of acquired competencies in electronic portfolios. This 
activity should stimulate teachers to adapt their teaching style to the 
requests from students for feedback on products that the students would like 
to put into their portfolios. After three iterations of development in three 
consecutive academic years, however, the use of portfolios is still not 
successful. The reasons why are discussed in this paper, and future steps to 
be taken are suggested. (Author) 
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Q , Abstract 

W At the University of Twente a new curriculum on educational science and technology has been introduced. That occasion was 

used to try to develop an apprenticeship model in which the students are regarded as young professionals from the very 
beginning. In that model the students are expected to govern their professional growth by actively collecting evidence of acquired 
competencies in electronic portfolios. This activity should stimulate teachers to adapt their teaching style to the requests from 
students for feedback on products that the students would like to put into their portfolios. After three iterations of development in 
three consecutive academic years, however, the use of portfolios is still not successful The reasons why are discussed and steps 
to be taken are suggested. 



The Project 

The Faculty of Educational Science and Technology at the University of Twente in the Netherlands is educating educational 
designers with specializations in seven directions: curriculum development, instructional technology, instrumentation (media, 
computers, and internet in education), educational organization and management, Human Resources Development (HRD), 
educational testing, and social science research. The Faculty intends to switch to a competency -oriented curriculum for its three- 
year Bachelor program. 

The term competency is partly to be understood in terms of the knowledge and skills that comprise the competency profile 
of a profession and partly as the ability “...to operate in ill-defined and ever-changing environments, to deal with non-routine and 
abstract work processes, to handle decisions and responsibilities, to work in groups, to understand dynamic systems, and to 
operate within expanding geographical and time horizons” (Keen, 1 992). Knowledge and skills can be assessed in traditional 
ways. The kind of behavior as described by Keen is more difficult to observe. This at the same time causes a problem in terms of 
the extent to which competency in Keen's terms should dominate the whole philosophy and organization of the program. To get 
around this problem, competency development was seen as an activity that can take place in concert with doing traditional 
coursework, as long as the students would be able to handle the related learning experiences from a meta-cognitive level that 
complies with Keen's definition of competent behavior. 

The use of portfolios offers a possibility to build evidence of the development of competencies. The main attraction of 
portfolios is formed by its potential to assess progress and process as essentials for learning (Saywer, 1 994). In the new 
curriculum portfolios are introduced for self-assessment by the students as well as for monitoring their progress. 

Starting in the academic year 1 999, electronic portfolios were introduced to provide students with a tool for actively working on 
competency development. The introduction was monitored and the use of the portfolios was developed in a few steps aiming at 
answering the following question: 

"Do student portfolio's yield sufficient support to help students to direct their own development in a competency-oriented 
curriculum, and if so, under what conditions?" 



The literature shows growing consensus that educational reform efforts are doomed to fail unless the teachers' cognitions, 
including their beliefs, intentions, and attitudes, are taken into account (Haney, Czemiak, & Lumpe, 1 996). The introduction of 
student portfolios is not only planned to serve the purpose of the students, but also as a tool for the implementation of the new 
curriculum. The latter relates to the readiness and willingness of teachers to adopt a new approach, for which the following 
question should be answered: 



"Does student -directed collection of evidence of professional growth influence teaching style of academic staff?" 

The educational concept "initiation in the academic profession" 

The new curriculum approach is based on a concept that is labeled: "initiation in the academic profession" (Verhagen, 2000). 
It is an apprenticeship model in which the students who enter the university directly from secondary school are regarded as young 
colleagues from the very beginning. A central principle of the. concept is that the interaction between students and teachers should 
take place in a professional context as much as possible instead of interaction in an instructional context. Assignments should as 
much as possible mirror professional practice to help students to develop professional behavior. The result of a literature 
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assignment should take the form of the literature part of a scientific article; communication products of design assignments such 
as proposals, budget estimates, blueprints or evaluation reports should take a form that would be appropriate to present to clients; 
and so forth. Teachers should primarily be regarded as experts who enable the development of academic insights by scientific 
discourse. 

To make this work, a new approach of mentorship has been introduced together with the new curriculum. Each student 
becomes a member of a mentor group that is chaired by a staff member of the faculty. A mentor group consists of about 12 
students from all three years of the Bachelor program, about four students from each generation. The mentor groups provide a 
social structure in which the master (the mentor/staff member) and the experienced students from the second and third year help 
the first year students (the learners) to be initiated in the professional culture of academic professionals. In these groups the 
discussion of personal development towards becoming a competent professional is a standing issue. The group members are 
required to collect evidence of professional growth that lends itself for discussion with the mentor and in the mentor groups. 

The students collect the evidence of their professional development in their electronic portfolios. Each portfolio consists of 
four parts: (a) An introduction of the owner (student), (b) a text based curriculum vitae, where the student is expected to put 
personal information, information about his or her school career, information concerning professional development in or outside 
of the official program (like having a job in the field) and other information such as involvement in sports and hobbies; (c) an 
archive, where the student puts evidence of his or her professional development during the study; and (d) a showcase where the 
student chooses to present a selection of his or her best work. The archive is the central tool for the student to perform self- 
assessments about specific accomplishments as well as to reflect on personal development in general. Self-assessment is 
considered to be a form of metacognition that is essential for self-regulation (Simon & Forgette-Giroux, 2000). 

Additional features of the new curriculum are that the development of information and communication skills is integrated 
into courses and that all courses are organized using a Web-based course management system (TeleTOP, a home-made Lotus 
Domino application). 

The implementation of the new approach requires a substantial change of teacher behavior. Most teachers are used to teacher- 
controlled instructional formats. Some teachers, however, share the philosophy of the new curriculum concept. They should act 
as the pioneers and early adopters who provide a critical mass of authentic professional tasks that allow student to develop the 
necessary skills for self-regulation of their academic education. These tasks and the interaction in the mentor groups are expected 
to shape the attitude and abilities of the student into characteristics of self-reliant young professionals. The extent to which this 
will appear to be true will answer the first research question. 

Approaching the students in such a way that they perceive the need to adopt a professional attitude is expected to cause 
students to work conscientiously on compiling evidence of their professional development in the form of portfolio products. 
They then will seek feedback of the teachers on their work and ask for comments on the added value of assignments for the 
objectives of a course and in the framework of the competency profile of the Bachelor program as a whole. It is expected that this 
behavior will influence the teaching style of the academic staff and will help teachers who are reluctant to invest in the new 
educational concept to move in the desired direction. The extent to which this effect occurs, will answer the second research 
question. 

The first experiences 

Preparing for the academic year 1999-2000 

The principles of the new approach have been presented to the teaching staff on several occasions, at first to estimate whether 
the concept was appealing to them. The overall impression was sufficiently positive to start the preparation of the introduction of 
the new approach in the program. A few months before the academic year 1999-2000, a group of student -friendly staff members 
was invited as mentors. Together with the faculty management they developed procedures and a related manual to start the new 
mentor groups. Involving the mentors at this stage resulted in their ownership of the concept for the new mentor groups and the 
way of working in those groups. Elements of the approach are that competency development was related to three major roles of 
professionals in our field: designer, researcher, and consultant; and that explicit attention should be paid to generic competencies 
such as planning, self management, interpersonal skills, communication, and academic reflection. In respect to the individual 
development of the students, a list of products that should be collected in the portfolios was specified. They concern results from 
assignments in courses that may be considered as evidence of acquired knowledge and skills, thus contributing to the competency 
profile of the student. Monthly professional meetings of the mentor groups were planned to discuss progress. During these 
meetings also attention was paid to the quality of the program as experienced by the students. It was expected that the students in 
their role as beginning educational designer should be interested in strengths and weaknesses of the courses in which they 
participate. The input from the mentor groups was also considered as valuable for the formal evaluation of running courses. The 
teaching staff was informed about the intended approach and invited to work accordingly. 

Outside the mentor groups, information and communication specialists developed their curricula in close cooperation with 
teachers from selected courses to arrive at the integration of relevant tasks and assignments in the different courses. 

Results from the academic year 1999-2000 

The mentor groups appeared to be handicapped by the fact that it was the first year and thus only first -year students were 
members. The monthly meetings failed also because the students had such a close contact with each other throughout the week, 
that no substance to discuss remained for the meetings. Course evaluations became a formal ritual with no real impact. 
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Teachers and students appeared to behave more traditional than expected, leading to much interaction in an instructional 
context and little in a professional context. The instructivistic teaching style in many courses appeared a dominant factor in 
shaping student behavior. Instead of working on a professional attitude that complies with the model of the students as young 
colleagues, the students felt that they went to school to take lessons and make tests. 

Moreover, due to technical problems portfolio software was introduced to the students at a late stage (the end of the first 
semester). Students had then to go back to already completed courses to find the required products for their portfolios. It was 
unlucky that for unclear reasons they appeared not to be informed about the list of required products that existed from the 
beginning of the academic year. And when they learned about the list, several students became annoyed because they consider 
that list as contradictory to the concept. If the portfolios are tools for governing one’s own learning process, they should be able 
to decide by themselves what to put into it. At that time, so much was unclear, that most students failed to work with the 
portfolios in a proper way. In conclusion, the electronic portfolios were hardly used. 

The only thing that really worked was the integration of information and communication skills into courses. The carefully 
developed set of tasks on information and communication skills made the students acquire the related skills every time they 
needed them for the assignments in the courses. A literature assignment in a course on pedagogy was used to teach them how to 
find literature, an assignment to write a paper for an other course was used to explicitly pay attention to writing skills, and so on. 

2000- 2001: Some changes 

The insight was developed that students should not be forced to put products in a portfolio, because this is contradictory to 
self-management. The students, who were interested in the desired approach, told that to us more than once and they were right at 
this point. The mentors were asked to guide the students in developing self- management skills, using the electronic portfolio as a 
discussion platform. 

Now that students of two study years were member of the mentor groups, activities were specified that could bring the 
concept to life. The elderly students could now introduce the new students to all kinds of procedures and habits in the faculty. 
And group discussion could now aim at points of interest for which the vision of both the first year and the second-year students 
was relevant. The number of official meetings, however, was reduced to seven. This measure was taken to avoid the problem of 
too few subjects for discussion that came with the monthly meetings in the first year. 

Changing the teaching style of teachers towards competence development seemed not to be possible directly. So a major role 
for the mentor was envisioned given the character of the guild model in the mentor groups. The mentors were asked to work with 
the students on helping them to use their portfolios for self-governance. 

Results from the academic year 2000-2001 

This time the mentors started to resist to the idea that they should work with the students in such a way that the students 
would develop the metacognitive skills to monitor their own professional growth with the electronic portfolio’s as the basic tool. 
They argued that the curriculum and the way in which the courses are taught, should have this effect. 

Again, the portfolios were hardly used although the software was now available almost from the beginning. But the early 
start had also a disadvantage. The students were introduced to the portfolio software in a workshop where to technical skills were 
practiced without real products to put into the portfolio. The first products that would be suitable had still to be produced in the 
courses that just were started. By the time that the portfolios could be used, most students had forgotten how to do that. As there 
was this time no list of required products, only few students appeared to motivated to start filling their portfolios. Many other 
students, however, appeared not to be able to decide what they could put into the portfolio. Partly the reason is that they appeared 
to be very critical of their first-year products, considering their own work as real beginners work that is not worth to be put into a 
portfolio. 

The cooperation between students from different years appeared to be one-sided. Only in the beginning of the academic year 
were the older students active when introducing the new students. Subjects of mutual interest to old and new students were not 
identified. The new students could also not bring anything of relevance for the older student. The question: “ What are my 
benefits?” was hard to answer for the older students. 

Still, the students were positive about the meetings of the mentor groups. During the meetings the students discussed general 
information with their mentor and with each other and they used the meetings as a platform to complain about organizational or 
educational problems in the faculty. They could speak freely about anything, which gave the meetings an ‘T am not alone with 
this” function. 

Putting students in control 

2001- 2002: A last chance for portfolio’s? 

Gradually it becomes clear that the basic philosophy of the concept “Initiation in the Academic Profession” does not really 
settle in the faculty. The belief that the concept is worthwhile is reason for a third attempt. This is where the approach was 
developed that is the reason for this paper. Again the idea is that the students should develop initiative in using their portfolios as 
a tool for collecting evidence of their professional growth. In the first week of the academic year, they were trained to use their 
portfolio’s in two ways: technically to learn how to put elements in the portfolio and how the manage the portfolio; and 
conceptually on how to use the portfolio for monitoring and managing professional progress. An adapted guide for the mentor 
groups explained the purpose and the philosophy: It was recognized that not all students are ready for this kind of metacognitive 
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activity. They are therefore allowed to use or not use their portfolios for self-management. Filling the archive with products of 
courses, however, is this time required to maintain basic portfolio skills until the moment that the student is ready and willing for 
the intended use. 

The results so far 

The number of students that works with the portfolios in the intended way is negligible. There are just a few students who 
work with their portfolios. These students are mostly using the archive function just for their own purposes and not to reflect on 
what they have done in past periods. Also most mentors still don’t use the portfolios for the individual meetings with the student. 
Some of them do, but they leave no room for self-directedness by the students because they require the students to fill the 
archive. 

In the meantime, the discussion about the usefulness of portfolios has become an issue in a broader perspective. Students who 
are following the old curriculum (from before the introduction of the new educational concept) start asking for their own 
portfolio’s for making overviews of products that they collect in courses during their study. So these oldest students see the 
purpose and the advantages of portfolios from a need for systematically archiving products. Regrettably, however, when 
providing them with the portfolio software, they don’t find the time to really do it. It seems a similar phenomenon as with the 
staff members who like the idea of the new educational concept, but do not really change their methods to comply with it. 

When student and staff members are asked what they think about the portfolio idea, they are almost all very positive. But still, it 
did not work out. The project fails and we have to find out what we may learn from it. 

Discussion 

Why portfolio’s can be a success 

When looking for a field where portfolios do work, the field of Human Recourse Development (HRD) is an obvious one. 
Self-responsible adults, who have a job and related responsibilities, benefit from individual learning arrangements that are 
reported by collecting evidence of achievements in a portfolio. The use of (electronic) portfolios is in that context appropriate 
because the learner has sufficient metacognitive (and computer) skills to use the portfolio tool properly. The educating agency 
(training department, external course provider, etc.) shares the philosophy of competency -based education and is therefore open 
to assessment on the basis of individual portfolios. This context is essentially different from the university situation where young 
students who enter the university directly from secondary school, do not have the maturity, the experience and the interest to 
work along these lines. This is the starting point for discussing why portfolios may fail. 

Why portfolio’s fail 

Portfolio’s fail when the students don’t see the value. Portfolio proponents tend to deny the psychological developmental 
stage of the students. But many (young) students are not prepared or willing to look at themselves in the metacognitive way that 
is required for proper dealing with portfolios. Further it seems that the spontaneous fun in studying theory is hampered by precise 
questions about the requirements that have to be fulfilled for a competency -based curriculum. A student, who is really involved 
in a subject, has to make a severe mental switch when he or she has to step outside that subject to analyze on a metacognitive 
level whether what he or she is doing is a contribution to the development of competence. 

And even when students see some value, for instance for building a comprehensive archive of their work during their study, 
they may misinterpret the function of the archive by denying products of which they are not very proud, like the first-year 
products when they feel themselves still beginners. To reflect on professional growth, however, these products are needed for 
reasons of comparison with later accomplishments. Proper guidance of the students by mentors could help, but this requires that 
the mentors are convinced of the value of portfolios as a tool for monitoring progress. In our case, we were clearly not able to 
motivate the mentors in this sense. 

And there is also a very practical reason why portfolios may fail. That is when the software causes problems. The system 
may need to many steps for simple tasks, the server may too often be too busy, and so on. In our case, several technical 
limitations did for certain not stimulate the use of the portfolios. 

Portfolios also fail when the teachers fail to adjust their teaching accordingly. Teachers are mainly prepared to carry out a 
well-defined course. When they have to step beyond the concrete patterns to adjust themselves to individual trajectories, many 
teachers fail to comply with that fact. The idea that a concept such as “initiation in the academic profession” can be put into 
practice outside the courses by regarding the traditional courses as occasions for gathering portfolio products that are used by the 
students and mentors, does not work. The concept and the use of the portfolios have to be operationalized within the courses. 

Where to go from here? 

Who wants to succeed in an effective learning process, ought to be able to coordinate his own learning process (McCombs, 
1988). In order to make the concept “initiation in the academic profession” successful, we will have to arrange a situation in 
which the students will be helped to get ready to do so. And this has primarily to happen in courses, while the new mentor groups 
may have a support function. The developmental readiness of the students has to be taken into account. Alexander (1995), for 
example, mentions three stages for the evolution for the learner. In the habituated stage, the student has a diminutive knowledge 
level. Having just a little domain-specific knowledge the student appeals on common strategies. The second stage is the ability or 
competency level. The students get more comprehensive and coherent knowledge of the subject and there is a change the student 
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will select the correct strategy for the specific situation. The third stage is the expert level. At this stage the students have ample 
knowledge of domain specifics, are ready to regulate themselves, and are able to add new knowledge to the domain. These stages 
ask for a curriculum line in which each stage has a logical place. This leads in our case to a choice for courses in which the three 
stages have a natural place: the series of courses about design methodology that runs form the first to the third year. This choice 
IS inspired by the Design Studio as it works at the Master’s level at the University of Georgia at Athens (Rieber, 2001) The 
principle as it will be tried in our program is that third-year students will take responsibility for design assignments while second- 
year students will act as helpers for specific tasks that need already proper workmanship (like carrying out a literature study or an 
students will be used for very concrete tasks for which it is not necessary to be very knowledgeable 
about the specific domain. In this stream, the principles of the concept “initiation in the academic profession” may be fully 
exploited, together with the use of portfolios. Next to and in balance with this stream, theory courses may still be taught in more 
traditional ways as long as all teaching complies with the seven principles for good practice in undergraduate education as listed 
by Chickering en Gamson (1987, quoted by Chickering en Ehermann, http://www.aahe.org/technology/ehrmann.htm): (a) good 
practice encourages contacts between students and faculty, (b) good practice develops reciprocity and cooperation among 
students, (c) good practice uses active learning techniques, (d) good practice gives prompt feedback, (e) good practice 

emphasizes time on task, (f) good practice communicates high expectations, and (g) good practice respects diverse talents and 
ways of learning. 



After the lessons learned with our attempts to introduce electronic portfolio’s faculty wide, we hope that the more modest 
approach for introducing portfolio’s and competency -based learning in the design stream of our program, will appear to be the 
right step to help to initiate our students into the academic profession. 
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